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Odyssey Reading 

No studies of Odyssey Reading that fall within the scope of the Adolescent Literacy review protocol 
meet What Works Clearinghouse (WWC) evidence standards. The lack of studies meeting WWC 
evidence standards means that, at this time, the WWC is unable to draw any conclusions based 
on research about the effectiveness or ineffectiveness of Odyssey Reading on adolescent learners. 
Additional research is needed to determine the effectiveness or ineffectiveness of this intervention. 

Program Description 1 

Odyssey Reading, 2 published by CompassLearning®, is a web-based K-12 reading/language arts program 
designed to allow for instructional differentiation and data-driven decision making. 3 The online program includes 
electronic curricula and materials for individual or small-group work, assessments aligned with state curriculum 
standards, and a data management system that allows teachers to develop individualized instruction and assessment 
tools to track individual student and classroom performance. Odyssey Reading can be used as a stand-alone 
curriculum or as a supplement to other reading and language arts curricula. This report focuses on Odyssey 
Reading programs in grades 4 and higher. 4 Odyssey Reading for students in grades 4-8 provides instruction in 
phonics, context, decoding, and comprehension. In later grades (9-1 2), the curriculum features a number of fiction 
and nonfiction texts that students read and analyze. The curriculum uses technology to create a “person” who 
simulates a conversation between the user and the software program. Younger students (grades K-2) also have 
decodable readers to assist with understanding the text and writing tools, a place where they can write, read, tell 
stories, and illustrate their thoughts. These activities aim to allow students to build upon prior knowledge and apply 
the tools and strategies that they are learning as part of the program. 

Research 5 

The WWC identified 27 studies of Odyssey Reading for adolescent learners that were published or released 
between 1 989 and 201 1 . 

Five quasi-experimental studies are within the scope of the Adolescent Literacy review protocol but do not meet 
WWC evidence standards. 

• Four studies do not establish that the comparison group was comparable to the treatment group prior to the 
start of the intervention. 

• One study has a confounding factor: Only one classroom was assigned to the treatment group, which makes 
it impossible to attribute the observed effect solely to Odyssey Reading. 6 

Twenty-two studies are out of the scope of the Adolescent Literacy review protocol. 

• Eight of these have an ineligible study design: a case study or a single group pre-post design. 

• Seven studies are literature reviews or meta-analyses. 

• Two studies do not examine the effectiveness of an intervention. 

• The remaining five studies either do not report an eligible literacy outcome or feature students who are not 
in grades 4-12. 
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Endnotes 

1 The descriptive information for this program was obtained from a publicly available source: the program’s website (http://www. 
compasslearning.com, downloaded June 2011). The WWC requests developers to review the program description sections for accu- 
racy from their perspective. The program description was provided to the developer in July 201 1 . Further verification of the accuracy 
of the descriptive information for this program is beyond the scope of this review. The literature search reflects documents publicly 
available by August 201 1 . 

2 The web-based Odyssey Reading program was initially delivered under the Jostens’ brand. Both programs, Odyssey Reading and 
Jostens Integrated Learning instruction, are included in this report. 

3 CompassLearning® offers three Odyssey Reading programs as a continuous sequence from kindergarten through high school: 
Odyssey Reading/Language Arts Grades K-2, Odyssey Reading/Language Arts Grades 3-8, and Odyssey High School. Odyssey 
High School is the name used for the intervention in grades 9-12. 

4 Studies of Odyssey Reading conducted in kindergarten through third grade are outside of the scope of the Adolescent Literacy 
review protocol. 

5 The studies in this report were reviewed using WWC Evidence Standards, Version 2.1, as described in the review protocol for 
Adolescent Literacy topic area, Version 2.0. The evidence presented in this report is based on available research. Findings and 
conclusions may change as new research becomes available. 

6 Please see the WWC Procedures and Standards Handbook, Chapter III for additional details on confounding factors. 

Recommended Citation 
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Glossary of Terms 

Attrition 

Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Extent of evidence 

Improvement index 

Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Rating of effectiveness 

Single-case design 
Standard deviation 



Statistical significance 



Substantively important 



Attrition occurs when an outcome variable is not available for all participants initially 
assigned to the intervention and comparison groups. The WWC considers the total attrition 
rate and the difference in attrition rates across groups within a study. 

If treatment assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons between outcomes and studies. 

A study is eligible for review and inclusions in this report if it falls within the scope of the 
review protocol and uses either an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

An indication of how much evidence supports the findings. The criteria for the extent of 
evidence levels are given in the WWC Procedures and Standards Handbook (version 2.1). 

Along a percentile distribution of students, the improvement index represents the gain or 
loss of the average student due to the intervention. As the average student starts at the 
50th percentile, the measure ranges from -50 to +50. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust the 
statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which subjects are assigned to 
treatment and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which investigators randomly assign 
eligible participants into treatment and comparison groups. 

The WWC rates the effects of an intervention in each domain based on the quality of the research 
design and the magnitude, statistical significance, and consistency in findings. The criteria for the 
ratings of effectiveness are given in the WWC Procedures and Standards Handbook (version 2.1). 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 

The standard deviation of a measure shows how much variation exists from the average. A 
low standard deviation indicates that the data points tend to be very close to the mean; a high 
standard deviation indicates that the data points are spread out over a large range of values. 

Statistical significance is the probability that the difference between groups is a result of chance 
rather than a real difference between the groups. The WWC labels a finding statistically significant 
if the likelihood that the difference is due to chance is less than 5% (p < 0.05). 

A substantively important finding is one that has an effect size of 0.25 or greater, regardless 
of statistical significance. 



Please see the WWC Procedures and Standards Handbook (version 2.1) for additional details. 
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